Indonesian Journal of Electrical Engineering and Computer Science 
Vol. 12, No. 2, November 2018, pp. 548~555 
ISSN: 2502-4752, DOI: 10.1159 1/ijeecs.v12.12.pp548-555 O 548 


Ambient Task with Stimulus-Response Compatibility Concept 


for Guiding Alzheimer’s Actions 


N. W. Basharudin, Ku Azir K.N.F, P.Ehkan, A. M. Khairuddin 
Embedded, Networks and Advanced Computing Research Cluster (ENAC), 
School of Computer and Communication Engineering, Universiti Malaysia Perlis (UniMAP), Perlis, Malaysia 


Article Info 


Article history: 


Received Apr 9, 2018 
Revised May 20, 2018 
Accepted Jul 11, 2018 


Keywords: 


Stimulus-response 
compatibility 
Ambient Intelligent 
Stroop task 
Alzheimer’s disease 
Flashcard game 


ABSTRACT 


Alzheimer is one of the diseases that can cause the patient to act in different 
and unpredictable ways. This disease destroys memory and thinking skills 
and, eventually, the ability to carry out the simplest tasks. Advanced in 
technology actually can help them be more independent doing their routines. 
Hence, this study was conducted to investigate the relationship between 
stimulus-response compatibility (SRC) and ambient intelligent elements for 
Alzheimer’s patients. The aim of this study is to obtain a strong answer on 
how ambient elements (stimulus) such as ambient colours, lights and pictures 
can assist Alzheimer’ patients in daily living activities. For this reason, we 
developed a flashcard game based on ambient colours and pictures to achieve 
two objectives which are (i) to test the ambient’s factors that highly effect the 
responses of Alzheimer’s patients and (11) to observe the response towards 
the given instructions. This experiment has been tested to 10 participants 
with mild-stage of Alzheimer’s disease. The findings of this experiment 
show that the patient response more towards ambient colours compared to 


the ambient pictures. 90% of respondents capable to complete the task 
correctly and quickly based on ambient colours. In addition, the experiment 
proved that the participants have no difficulty in order to understand the 
instructions given to them for completing the tasks smoothly. 
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1, INTRODUCTION 

The total population of Malaysia is estimated at 31.7 million persons recorded by Department of 
Statistics Malaysia until 2016 [1]. Distribution of population by age group showed population aged 65 years 
and above was increased by 0.2% for 3 years since 2014 [1]. The statistic also recorded that senior citizens 
aged 60 and above made up 2.8 million or 9% of Malaysia’s population of 31 million until 2015 [2]. 

Dementia is probably one of the more feared illnesses of old age. As the number of senior citizen 
increases, the disease also increases such as Alzheimer. Those people over the age of 65 and 80, from 5% and 
20% of them respectively have a risk of getting cognitive impairment. This disease causes patients 
experienced short-term memory loss. As time goes on, the person’s ability to remember, reason understand 
and communicate will progressively decline. Their memory loss may result in them forgetting to perform 
routine activities like eating or cleaning up or insisting on doing these activities repetitively because they 
have forgotten they have already done them. Therefore, the percentage of senior citizens with Alzheimer’s 
disease relying on people around them is increasing due to they're incapable to handle routine activities by 
themselves. Women, Family and Community Development Minister said the government has been making 
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various preparations to ensure this group of senior citizens would still be healthy, productive and able to 
contribute to society. One of the ways to support themselves is through ambient assisted technology. 

Ambient intelligent can be defined as a sensitive and adaptive environment towards the action of 
people and object and cater their needs. Ambient assisted living (AAL) technology can focus on developing 
an intelligent environment for monitoring routine activities of Alzheimer’s patients by using the ambient 
intelligent. Ambient intelligent is the tremendous solution for supporting people such as Alzheimer’s patients 
living independently in the home by becoming sensitive and adaptive environments towards the actions of 
people and objects and cater their daily needs ambient technologies assisted living user’s activities by 
capturing and monitoring the daily activities that provide non-intrusive and privacy-friendly. 

In this work, we describe an experiment of implementation of ambient intelligent elements with 
stimulus-response compatibility (SRC) concept. SRC refers to actions performing by people is more accurate 
and quick when mappings of stimuli to responses are concurrent than when they are non-concurrent [3]. This 
work conducted based on two objectives which are (1) is to identify either ambient colours or ambient 
pictures give high influenced towards Alzheimer’ actions while the (2) is to analyze the level of 
understanding against instruction delivered to patients in order to complete the task. These objectives had 
verified with simple flashcard game model consisting a combination of ambient colours and ambient pictures 
which is functioning to observe the relations between displays and controls which is the reaction of patients 
based on a specific instruction given. The idea of creating this experiment is inspired by developing a current 
user-friendly environment with applying ambient technology to assist Alzheimer’ behaviours in a certain 
situation. These ambient intelligent elements are tested in the kitchen workspace. 

The kitchen becomes the main focus because it is where many activities that are key to autonomy 
performed as kitchen involved tools, actions and multiple activities at the same time. Such as cooking while 
washing cloth and cooking while baking, these multiple activities can create possible wrong actions which 
will lead to the proposedly accident. Therefore, with the ambient light it 1s expected that when it been 
implemented in the kitchen, it can recognize possible unwanted actions. For example, misapplication of a 
spoon to break ice cubes while it is actually for food feeding or scoping purpose, another misapplication uses 
a knife as can opener cover while it is for cutting purpose. Ambient technology also approaches based on 
artificial intelligence techniques to distinguish an abnormal activity in the kitchen. 

This paper has presented the SRC concept and ambient elements that had applying in the 
flashcard game for Alzheimer’s patients. The fundamental of Stroop task study had implemented in flashcard 
game. The final result demonstrated ambient colours highly influenced actions of patients, leading to much 
faster and accurate responses. The benefit of this study is to get strong confirmation that the ambient 
elements can help the patient in guiding them to complete a particular activity. Next, will encouraging 
patients be more independent at home especially in the kitchen with ambient intelligent technology. 


2. RELATED RESEARCH 

The uses of AAL for monitoring activities and assisting a human in the smart environment have 
attracted tremendous research interests in recent years. AAL provides important development of innovation 
in information technology that should be can improving the quality of life and autonomy with the smart 
environment in house, office, vehicle, street and shopping mall. The smart environment can help people in a 
wide range of tasks by learning of behaviours which are a major recent advancement in the area. 

Activities of people can be monitored to detect abnormal activities to prevent undesired 
consequences before they become critical [4]. SmartMind is one of the systems that create for Alzheimer’s 
patients to describe their normal living habit besides aiming to handle emergency situation happened in living 
room [5]. An Ambient Information Systems help elders to manage their medicine by reminding, guiding and 
motivating them to medicate [6]. 

The concept of SRC is great ideas that have many basic and applied implications for human 
information processing and performance. The spatial arrangement between stimulus-response (S-R) refers as 
compatibility when relation between displays (stimulus) and controls (response) is direct [7]. Compatibility 
effects has found to be influence performance in various situations such as Stroop task. Stroop task is a 
paradigm of the colour-word task, participants have to identify the ink colour of a colour word printed easily 
and quickly in time. There are two methods which are colour-word matching Stroop task (cwmStroop) and 
Go/NoGo tasks in a clinical test of the attention and inhibition abilities of children with developmental 
disorders [8]. A study of stimulus and responses conflict in colour-word Stroop task had done by combining 
event-related brain potentials (ERPs) and electro-myography (EMG) [9]. Stroop effect in pure blocks had 
used in the assessment of inhibition and goal maintenance in persons with Alzheimer’s disease, mild 
cognitive impairment, healthy older adult and younger adults [10]. The concept of Stroop task is closely 
related to the flashcards experiment goal in this paper as it is involving flashcard colours and words tested. 
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SRC creates naturalness of the connection between stimulus and the associated response that will 
contribute to the fast reaction time in compatible S-R pairs. The combination of ambient intelligent elements 
and SRC concept in the smart environment can create a system to assist living activities of Alzheimer’s 
patients. Furthermore, through stimulus and response compatibility effects, we can study and analyse their 
behaviours patterns easily. Therefore, be prepared if any autonomous actions happened the systems will 
guide them back to correct and proper actions that should be performed. 


3. METHODS AND MATERIALS 

At the early stage of the research, we applied ambient flashcard game as part of monitoring purpose 
of Alzheimer’s understanding of meaningful of colour. The purpose of using flashcard is to utilize the 
patients by learning the materials and keeping the information at the forefront of mind also with a little 
spatial testing. It is suggested, that flashcard game will be able to help patient’s brain focus on the material 
and effectively keep it available for them to recall when the next test comes up. 

The game was conducted at The Dementia Society (TDS) Perak. The participants that involved in 
this experiment are had mild (early-stage) Alzheimer’s disease which is during this stage patient’s activities 
and social life is disturbed due to memory troubles. There are 10 participants (average age 82 years old) 
participated in this experiment, with 9 of them are Chinese and 1 Malay. These include 8 male and 2 female 
participants. All participants have no difficulties regarding speak and move around on their own although 
sometimes they forgot what actually they did. They are all showing interest to participate in the experiment. 

Figure | shows one of the participants 1s undergoing the flashcards experiment. In the colour-word 
of Stroop task, participants need to name the ink colour of word by ignoring the meaning of the word, which 
is they have to refer to the colours printed in a corresponding colour (the word blue printed in blue ink) or in 
non-corresponding colour (the word red printed in green colour). For the flashcard game, participants need to 
match cards of (1) picture-picture by ignoring the colour of cards; (41) colour-colour by ignoring the pictures 
on the cards; (111) picture-word (English words) by ignoring the colour of cards; and (iv) word (Malay 
words)-picture by ignoring the colour of cards. 

The experiment took place in a quiet environment which is in the activity room of TDS. At very 
first, each participant was briefed and demonstrated, also instructions are given to the participants verbally by 
the instructor at the beginning of the experiment. For each condition, the participant had prepared to seat on 
the chair facing the board of stimuli pile flashcards on the table. 

There are two set of model, which are the stimuli pile and the responds pile. The stimuli pile consists 
of flashcards which act as the stimulus for this experiment. Meanwhile the responds pile contains another and 
different set of flashcard which will be hold by the participants. These flashcard act as responds and ready to 
react accordingly to the instructions given. There are two cameras had installed from the front view and top 
view of the place of experiment. The front cameras for recording the expressions or body language of 
participants during the experiment while the camera from above is top viewing to monitor running 
experiments from start to finish. In addition, the time from instruction given was taken as time reaction and 
no specific time limitation for the participants to complete the task. 
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Figure |. Participant undergoing the experiment 


4. ACTIVITY TYPE 

This game consist of specific elements which are five ambient colours; blue, red, green, yellow and 
orange colours while ambient pictures had two groups consist of five pictures of fruits and utensils equipment 
as shown in Figure 2. 
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Figure 2. The ambient elements in the flashcards model 


A participant needs to complete four different tasks from this experiment. Briefly, the participants 
require to match the responds pile to the stimuli located in the stimuli pile. All the four tasks that need to 
complete is as follows: 


4.1. Picture — Picture: Participant is required to match the flash cards by referring picture on stimuli pile. In 
this task, participants require to match the responds pile of utensils pictures with the same pictures on the 
flashcards located inside the stimuli pile as shown in Figure 3. The purpose of this task is to observe the 
ability of Alzheimer’s patients recognizing the picture of utensils correctly. 


4.2. Colour — Colour: Participants is required to match the flash cards by referring card colour on the stimuli 
pile. This task required the participants to match the card of responds pile with the same colours as cards on 
the stimuli pile by ignoring the picture display on the card, Figure 4. It is purposed to observe either 
Alzheimer’s patient be able to identifying colours correctly or not. 


4.3. Picture — Word: Participants needs to place the flash card according of the correct name of fruits in 
English. Participants had asking to match the pictorial cards of fruits in responds pile to the correct cards 
with written word of fruit’s name on the stimuli pile as shown in Figure 5. From this task, we can observe the 
effectiveness of participant’ responses towards stimulus in the form of words. 


4.4. Word — Picture: Participants needs to place the flash card according of correct picture of the fruit as 
shown in Figure 6. Actually, this task is inversing from the Task 3 that needs the participants to match the 
pictorial cards of fruits (responds pile) with written word of fruit’s name in Malay on the stimuli pile. 





Figure 3. Task 1 - Experiment setup to match Figure 4. Task 2 - Experiment setup to match 
picture and picture colour and colour of cards 
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Figure 5. Task 3 - Experiment setup to match word 


and picture 
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Figure 6. Task 4 - Experiment setup to match picture 


and word 


The instruction for each task are as shows in Table 1 and the list of stimulus and possibilities 
responses that will react by participants for completing each instruction 1s listed in the Table 2. 


Table 1. Four different instructions of simple ambient task 


Instructions 


Do Instruction 1: Match the flashcard by referring to 


picture on the given model. 


Do Instruction 2: Match the flashcard by referring card 


colour on the given model. 


Do Instruction 3: Place the flashcard according of 


correct name of the fruit 


Do Instruction 4: Place the flash card according of 


correct pictures of the fruit. 


Match the flashcard 
Picture with Picture 
(utensil X utensil) 


Colour with colour 


Pictures with Words 
(Fruit X name of fruit) 


Word with Picture 
(Name of fruit X fruit) 


Stimulus Responses 
Utensil Utensil pictures 
pictures 


Colour card Colour card 


Fruit Name of fruits 
pictures 
Name of Fruit pictures 


fruits 


Table 2. List of Stimulus And Possibilities Responses 


Peeler 


Responses 


Fruit Juicer 


Stimulus 
Instruction 1 

Pictures 
Instruction 2 Colours 
Instruction 3 Pictures 
Instruction 4 Words 


Knife 


Cutting Board 
Glass 
Fruit Juicer 


Red 
Yellow 
Orange 
Blue 
Green 


Grapes 
Banana 
Orange 

Carrot 

Apple 

*Pisang’ 

‘Epal’ 

‘Lobak Merah’ 
‘Oren’ 
‘Angegur’ 


Pictures 


Colours 


Words 


Pictures 


Knife 


Glass 
Cutting Board 
Peeler 


Green 
Blue 
Orange 
Yellow 
Red 


‘Grapes’ 
‘Banana’ 
‘Orange’ 
‘Carrot’ 
‘Apple’ 
Grapes 
Banana 
Orange 
Carrot 
Apple 
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5. RESULTS 

The result of the test is analysed from understanding, accuracy and time taken for completing tasks. 
The results recorded based on reaction time of respondents which is in seconds (s). The reaction time of 
completing tasks divided by four interval of time, 1 to 30 seconds, 31 to 60 seconds, 61 to 90 seconds and 
exceed 90 seconds. These time interval shows an early hypothesis of the experiment, the increase the timing 
of participant completing the task, the higher probability participant facing difficulties to complete task. The 
difficulties faced by participants might be regarding their understanding of instruction given and lack of 
memory half-way in completing a particular tasks. Table 3 shows number of participants in completing all 
tasks within the time interval given. 


Table 3. Number of participant response to the tasks 


Timing Task 1 Task 2 Task 3 Task 4 
1 — 30 (Fastest) 2 2) 5) 2 
31 — 60 (Slow) 5 4 3 ph 
61 — 90 (Slower) 3 ] 2 2 
> 90 (Slowest) 0 0 0 4 


Figure 7 shows four different graph that obtain from the Table 3 shows result obtained for all tasks 
had conducted. Graph (a) recorded the results for first task which ask the participants to match the flashcard 
by referring to picture on the given model. There are 5 out of 10 participants perform Task | within 31 to 60 
seconds and only 2 participants successfully complete faster than others within 30 seconds. Whereas, graph 
(b) shows second task is matching the flashcard by referring card colour on the given model. The results had 
recorded 5 out 10 participants take less than half of minute (< 30 seconds) and 4 participants takes 31 
seconds to 60 seconds to complete Task 2. This is shows that ambient colours give high influence to the 
participant’s response rather than ambient pictures in their daily activity. 

On the other side, graph (c) displays that third task is placing the flashcard according of correct 
name of the fruit. This illustrates that 5 participants able to complete Task 3 within 30 seconds. This is 
demonstrating participants with Alzheimer disease have high capability to recognize what kind of pictures 
give to them. However, graph (d) present forth task is placing the flash card of fruit’s name in Malay 
according of correct pictures of the fruit. The result recorded that 4 out 10 participants take more than 90 
seconds and 2 participants for others time interval respectively to complete Task 4. With this result, proved 
that people with Alzheimer’ disease quite weak to read the Malay words and this contribute they taken longer 
time in order to complete a particular tasks. 

As we can notice from Table 4, all participants completed Task 1 successfully. All of participants 
perform very well in completing Task 1, this shows that Alzheimer’s people able to recognize pictures 
clearly. In contrast, there are participants failed to finish Task 2, only 9 participants did perform well. This is 
might cause of existing of confusion while they completing Task 2 for another one participant. For Task 3, all 
the participants successfully identify pictures clearly and match with word correctly. This proved that all 
participants did not have any problem regarding to recognize and remember pictures and picture’s names in 
English. Lastly, Task 4 shows that it is quite difficult for them to complete it due to difficult to read and 
understand the word in Malay also there 3 participants are unable to read. 

To summarize, all of the participants performed well in matching the flash card in Task | however, 
5 of participants took longer time to complete the task. On the contrary, result obtained from Task 2 shows 9 
participants correctly match the flash card according the instruction and 5 of them response very quick in 
time to complete. Therefore, we can concluded that colours is an element that can attract and influence the 
efficiency of Alzheimer’s responses in a particular activity. This conclusion is supported by Macleod, who’s 
concluded that performance in Stroop tasks is consistently better on congruent trials than on incongruent 
trials [11]. For instance, the word RED printed in red ink is congruent trials while, incongruent trials when 
the word RED printed in blue ink. The tasks which is involving of words for example, Task 3 and Task 4 
give low effect towards participants might be due to weakness in reading. We can say words is not suitable to 
implement in a particular ambient system in order to deploy for Alzheimer’s patients. The participants gives 
very positive feedback on the flashcards game. 


Ambient Task With Stimulus-Response Compatibility Concept for... (N. W. Basharudin) 


554 O ISSN: 2502-4752 





Task 1: match picture = picture Task 2: match colour -> colour 
~% | 
(a) fb) 
Task 3: match picture -> word Task 4: match word -> picture 
(c) (qd) 


Figure 7. Results of (a) Task 1: match between pictures. (b) Task 2: match between colour card. 
(c) Task 3: match pictures with words. (d) Task 4: match word with pictures 


Table 4. Number of participants response accurately to the instructions/task. Responses of 
participants to the tasks 


Task TRUE FALSE 
Task 1 10 0 
Task 2 9 1 
Task 3 10 0 
Task 4 5 5 


6. CONCLUSION 

From this experiment, we concluded that patient with Alzheimer’s able to understand and response 
toward instructions given. However, reaction time of their responses could be uncertain due to memory 
confusion in the middle of completing a particular task. From the result shows that responses by participants 
with Alzheimer’s disease in particular tasks is more influenced by colours. As most of them response with 
less time in completing Task 2 with 90% perform within 30 seconds only. Therefore, it is suggested that in 
the future, the smart kitchen environment can be build based on the ambient colours, text, pictures in 
monitoring their activities in the area. It is also suggested that these ambient elements can assisting them of 
giving the right sequence of activities if they did wrong. From this experiment, obtained that studying 
stimulus-response compatibility related on cuing effects typically effected reaction time and error rate 
performance of respondents. Designing digital interfaces are increasingly being used for a variety of purposes 
because they have the potential to facilitate effective and efficient interactions between humans and 
computers by applying concepts of SRC. Human factors specialists have recognized the importance of 
compatibility effects in designing interfaces since the earliest days of research in the field. The overall results 
should help industrial developing an effective and helpful interface in machinery of ambient intelligent for 
assisting patients in their daily living activities especially in the kitchen environment. 
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